Python: Add tool concurrency groups and sequential execution order for same-message calls - #7523
Conversation
GitHub Actions
Inactive
GitHub Actions
Inactive
GitHub Actions
Inactive
GitHub Actions
Inactive
agent-framework-automation
Bot
added
the
python
There was a problem hiding this comment.
Pull request overview
This PR adds framework-level controls to prevent same-message tool-call race conditions in the Python core by allowing (a) per-tool serialization via concurrency groups and (b) run-level sequential execution of all tool calls in a batch.
Changes:
- Added
concurrency_group: str | NonetoFunctionTooland the@tooldecorator, enabling sequential execution for tools sharing a group within a single message batch. - Added
tool_execution_order: Literal["parallel","sequential"]to chat options and function invocation configuration, and wired it through to the function-call execution layer. - Added unit tests covering concurrency-group serialization and sequential execution behavior.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| python/packages/core/agent_framework/_tools.py | Adds concurrency-group metadata, serializes it, and implements grouped/sequential function-call execution and option wiring. |
| python/packages/core/agent_framework/_types.py | Extends chat options with tool_execution_order to expose the configuration at the API surface. |
| python/packages/core/tests/core/test_tools.py | Adds tests validating concurrency-group handling and sequential tool execution order. |
GitHub Actions
Inactive
|
|
||
| # Bind one executor with the run's custom arguments, middleware, configuration, and session. | ||
| request_config = dict(self.function_invocation_configuration) | ||
| if tool_exec_order := mutable_options.pop("tool_execution_order", None): |
There was a problem hiding this comment.
Could we keep the popped dictionary instead of rebuilding it below? tool_execution_order is removed here, but mutable_options is recreated from the original options at _tools.py:3335, so the key still reaches super_get_response and is splatted into provider SDK calls, where it raises an unexpected-keyword TypeError whenever this per-run option is used.
| group_key: str | None = None | ||
|
|
||
| if execution_order == "parallel": | ||
| tool = tool_map.get(function_call.name) |
There was a problem hiding this comment.
What happens when these are resumed function_approval_response items? Their outer Content.name is None; the tool name is nested under Content.function_call.name, so every approved grouped call falls into a separate __ungrouped_* bucket and execute concurrently again. This bypasses concurrency groups on approval resume. Could this use _underlying_function_call(function_call).name instead?
Motivation & Context
Currently, the framework executes all tool calls requested in a single assistant message concurrently. While this is a great default for independent calls (like parallel document lookups), models routinely emit dependent calls in one batch (e.g., "write the file, then read it"). Because the tool author has no way to serialize these calls, dependent reads race the still-running writes, leading to "not found" errors and contradictory agent states.
This PR closes that gap by providing declarative, framework-level control over tool execution order, preventing stateful tool race conditions without relying on fragile, tool-side
asyncio.Lockworkarounds.Fixes #7386
Description & Review Guide
What are the major changes?
concurrency_group: strparameter toFunctionTooland the@tooldecorator. Tools sharing the sameconcurrency_groupexecute sequentially in call order within a message batch, while ungrouped tools remain fully concurrent.tool_execution_order: Literal["parallel", "sequential"]to_ChatOptionsBaseandFunctionInvocationConfiguration. Setting this to"sequential"forces all tool calls in a batch to execute one-by-one.tool_execution_orderchat option through theFunctionInvocationLayerdown to the execution engine so the setting actually takes effect at runtime.FunctionTool.to_dict()to ensureconcurrency_groupsurvives serialization, and added docstrings documenting the ordering guarantee (specifically requested in the issue).What is the impact of these changes?
This is fully backward compatible. The default behavior remains
"parallel"with noconcurrency_groupset, ensuring existing agents behave exactly as before. It provides tool authors a safe, declarative way to handle stateful dependencies.What do you want reviewers to focus on?
Please review the grouping algorithm in
_try_execute_function_call_groups(_tools.py). Specifically, verify that theordered_resultsarray correctly maps indices to ensure results are returned in the exact order the model requested them, and thatcontextvars.copy_context()is still applied correctly per-call to preserve agent span observability.Related Issue
Fixes #7386
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.